Towards a comprehensive testbed to evaluate the robustness of reputation systems against unfair rating attack

Evaluation of the effectiveness and robustness of reputation systems is important for the trust research community. However, existing testbeds are mainly simulation based and not flexible to perform robustness evaluation, and none of them is specifically designed to evaluate the robustness of reputation systems against unfair rating attacks. In this paper, we propose a novel comprehensive testbed by simulating three types of environments (simulated environments, real environments with simulated unfair rating attacks, and real environments with detected unfair ratings). The testbed incorporates sophisticated deception models and unfair rating attack models, and introduces several performance metrics to fully test and compare the effectiveness and robustness of different reputation systems. We also provide two case studies to demonstrate the usage of partial features of our proposed testbed.